Skip to content

FINAL clause speed: refresh for ClickHouse 23.2-26.10 - #194

Merged
BorisTyshkevich merged 1 commit into
mainfrom
docs/final-clause-speed-26.10-refresh
Nov 12, 2026
Merged

BorisTyshkevich merged 1 commit into
mainfrom
docs/final-clause-speed-26.10-refresh

Conversation

@BorisTyshkevich

Copy link
Copy Markdown
Contributor

FINAL clause speed article: changelog refresh + benchmarks

Updates content/en/altinity-kb-queries-and-syntax/altinity-kb-final-clause-speed.md from the ClickHouse changelogs (23.2 to 26.10), with every setting default and version checked against system.settings / system.settings_changes in official Docker images (25.8, 25.12, 26.1 to 26.10).

Corrections to the old article

  • Excessive-read fix is 23.3 (not 22.8); enable_vertical_final was off in 24.5 and on again from 24.6; use_skip_indexes_if_final_exact_mode added 25.5 (default 25.6); skip-index PK-intersection check removal is 25.11 (not 26.1); apply_row_policy_after_final default 1 in 25.12, 0 in 26.1, 1 from 26.2.
  • Behaviour verified across versions: JOIN with FINAL on the left table applies FINAL to all joined tables up to 26.5 and only to the left one from 26.6 (analyzer_compatibility_apply_final_to_all_joined_tables=1 restores); reverse-order FINAL read from 26.9; defer_partition_pruning_after_final changes results as described; final=1, PREWHERE-before-FINAL and row policy behaviour as described.

Benchmark 1: PR #121799 (skip index, exact mode), 26.8.7.19 vs 26.10.3.7

Host gpu-01 (20 cores, 62 GB), container --memory 12g --cpus 8, one version at a time, 8M rows, 1 warmup + 5 runs, mark/uncompressed caches dropped before each run (OS page cache warm). Query SELECT count(), sum(key) FROM t FINAL WHERE v = 1. Results equal use_skip_indexes=0 in all 18 configs; selected marks and read_rows identical on both versions. Medians, ms:

Parts (granularity 8) 26.8 26.10 Marks
5 397 57 4439
10 384 53 8433
20 377 62 16375
40 (PR layout) 417 103 32057
100 524 218 77570
index_granularity (40 parts) 26.8 26.10
8 417 103
64 125 98
512 132 132
8192 134 132
Selectivity (granularity 8, 40 parts) 26.8 26.10 Marks
1/200000 353 50 3645
1/20011 417 103 32057
1/2000 822 628 279398
1/200 1651 1547 961281

Other: index type (minmax 417/103, set 429/117, bloom_filter 425/114) does not matter; non-overlapping parts still gain (granularity 8: 176 -> 25; granularity 64: 28 -> 8); index column in the PRIMARY KEY: 24 vs 38 ms, both skip the recovery pass; merged parts + 20 fresh parts at granularity 64, 1/2000: 166 vs 153 ms; composite ORDER BY (a,b), 20 parts, granularity 64: 80 -> 24 ms. Baselines on 26.10 (PR layout): exact 103, use_skip_indexes_if_final_exact_mode=0 255, use_skip_indexes_if_final=0 1064; at 1/200 exact (1547) is slower than no index (1101).
Not explained: exact mode faster than approximate mode on 26.10, and the non-overlapping gain. My PR-layout run selected 32057 marks and 417/103 ms, not the PR's 65466 and 3.88 s/0.76 s. Recovery-pass time alone was not captured.

Repro (PR perf test):

CREATE TABLE t (key UInt64, ver UInt64, v UInt8, INDEX idx_v v TYPE minmax GRANULARITY 1)
ENGINE = ReplacingMergeTree(ver) ORDER BY key
SETTINGS index_granularity = 8, index_granularity_bytes = '10Mi';
SYSTEM STOP MERGES t;
INSERT INTO t SELECT (number % 200000) * 40 + intDiv(number, 200000) AS key, 1, intHash64(key) % 20011 = 0
FROM numbers(8000000)
SETTINGS max_threads = 1, max_insert_threads = 1, max_block_size = 200000, max_insert_block_size = 200000,
         min_insert_block_size_rows = 0, min_insert_block_size_bytes = 0;
SELECT count() FROM t FINAL WHERE v = 1 SETTINGS use_query_condition_cache = 0;

Benchmark 2: lazy materialization / lazy FINAL (PRs #101647, #110722, #110576), 26.4.5.143, 26.7.38.18, 26.8.36.3, 26.10.3.7

Host llmbox-01 (24 cores, 214 GB), container --memory 16g --cpus 8, max_threads=8, 1 warmup + 5 runs, medians, warm caches (no root: OS page cache not dropped). t_w: ReplacingMergeTree(ver) ORDER BY k, 4 unmerged parts x 2M keys, 3 x String(200). Combos: N = everything off, D = defaults (lazy materialization on), L = lazy FINAL on, LN = lazy FINAL on, lazy materialization off. Median ms N/D/L/LN:

Query 26.4 26.7 26.8 26.10
SELECT * WHERE f=7 LIMIT 10 84/88/143/114 125/29/49/134 106/28/26/118 118/29/28/115
SELECT s1 WHERE f=7 LIMIT 10 36/38/48/64 35/28/41/64 44/24/21/36 35/28/27/48
5 x 1KB, LIMIT 10 230/304/336/355 282/27/33/323 301/26/26/287 277/31/28/285
narrow k,f, LIMIT 10 31/28/47/53 34/29/55/52 26/25/26/29 27/26/26/27
f=7 LIMIT 1000 169/171/186/183 162/81/102/180 158/81/82/159 167/101/90/169
f=7 LIMIT 100000 173/170/187/187 161/161/170/171 157/157/160/159 168/164/165/166
single row id=, LIMIT 100 451/439/26/27 436/72/24/24 431/70/69/434 429/76/75/434
f<5000 (50%), LIMIT 100 36/35/201/195 31/19/171/170 34/18/18/32 31/22/23/36
ORDER BY k, SELECT *, LIMIT 10 340/50/71/374 312/52/81/361 298/52/48/288 87/31/30/81
ORDER BY ver (non-key), LIMIT 10 173/77/91/188 159/67/80/173 154/67/76/168 159/73/92/174

Read bytes on 26.8, SELECT * LIMIT 10: 899 MB (N) -> 98 MB (D). Lazy FINAL helped only for a contiguous key range (c=5, no LIMIT: 428 -> 24 ms on 26.7, 5143 -> 95 MB) and a unique id without LIMIT (432 -> 26 ms); no-LIMIT cost at 10% selectivity 175 -> 245-260 ms and at 50% 487 -> 635-665 ms; worst case 26.7 f<1000 LIMIT 1 18 -> 93 ms. Plan lines: JoinLazyColumnsStep over Limit -> Filter -> ReadFromMergeTree (FINAL: 1) (not present on 26.4, nor without a filter, nor with LIMIT above query_plan_max_limit_for_lazy_materialization = 10000; verified locally). Correctness: no wrong results or exceptions on any version for Nullable PK with NULLs, is_deleted with PREWHERE, hasToken text index (plan not captured).

Not tested / dropped

make_distributed_plan (private preview, needs stateless workers); 25.1/25.10 FINAL performance entries are changelog-only.

Tested only against official Docker images; nothing was built with Hugo locally (CI-pinned 0.128.2).

🤖 Generated with Claude Code

https://claude.ai/code/session_01D1KN5umrvcVcdzrL77H4k3

… and lazy FINAL notes

Fix wrong versions (23.3, 24.5/24.6 vertical FINAL, 25.5/25.6 skip indexes,
25.11, row policy defaults), add missing changelog entries, a skip-index
exact-mode note (PR 121799) and a lazy FINAL vs lazy materialization section.
Benchmarks and methodology are in the PR description.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1KN5umrvcVcdzrL77H4k3
@github-actions

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@BorisTyshkevich
BorisTyshkevich merged commit 3f92678 into main Nov 12, 2026
1 of 2 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 12, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant